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New concepts from nonequilibrium thermodynamics are used to show that Landauer's principle 
(-H ' can be understood in terms of time asymmetry in the dynamical randomness generated by the 

physical process of the erasure of digital information. In this way, Landauer's principle is generalized, 
showing that the dissipation associated with the erasure of a sequence of bits produces entropy at 
' the rate k^I per erased bit, where / is Shannon's information per bit. 
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O ' Landauer's principle says that the minimal dissipation accompanying the erasure of one bit of information produces 
an entropy equal to fee In 2 where fce is Boltzmann's constant [l[ . This important result is based on the observation 
that the processing of digital information is a physical process among others and should thus obey the laws of 
, K , thermodynamics. Landauer's result has therefore supported the idea that information processing devices working at 
temperature T should dissipate at least k B T ln2 of energy during an elementary act of information 0, [![• Later, 
Bennett's work clarified the point that Landauer's dissipation is the feature of logically irreversible operations on 
-4-J ■ data, i.e., operations whose inverse is not unique, such as data erasure [ij. Moreover, Bennett used Landauer's result 
to resolve Maxwell's demon paradox Since then, Landauer's principle has been explicitly verified in specific cases, 
for instance, in bistable potentials with white noise Q , for a bit in contact with a thermal reservoir 0] , or from coarse 
1 , ' graining in phase space [8| . 

Landauer's result has been verified using case studies and it is only recently that nonequilibrium statistical mechanics 
has been sufficiently advanced in order to reach its understanding in a general framework. Indeed, it has been recently 
established that the second law of thermodynamics finds its origin in the time asymmetry of the property of dynamical 
randomness, i.e., the temporal disorder that a physical process develops during its time evolution [3, ITol . [TTl H3 |. 
Dynamical randomness is characterized by the decay rate of the probabilities of the typical paths or histories followed 
i by the fluctuating process. The time asymmetry of the process can be tested by comparing the decay rate of the path 
[- — ' probabilities with the one of the corresponding time-reversed process. These decay rates are the so-called entropy per 
£N| , unit time 
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and time-reversed entropy per unit time 
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where u)\UJ2 • • • to n denotes a path. \x is the invariant probability measure of the equilibrium or nonequilibrium process, 
whereas p, is the invariant measure of the backward process with reversed nonequilibrium drives 0, [ToL [ill ]. The 
paths are observed every time interval r (considered as the unit time) and sampled into states u>j (with ] = ■■■ — 
2,— 1,0, 1, 2, • • •). These dynamical entropies may be considered as rates of production of information generated by 
the fluctuations of the process during its time evolution. In this sense, they characterize the dynamical randomness 
of the process. It has been established 0, [l(| El that the difference between these quantities gives the well-known 
thermodynamic entropy production according to 

AiS = k B (h R ~h) (3) 

where A{S is the entropy produced during the unit time r. The difference between the two entropies per unit time 
is the relative entropy between the forward and backward paths, which is known to be non-negative and to give the 
thermodynamic entropy production for instance in Markovian stochastic processes 0, [ToL [ll| . This result has been 
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verified in experiments on driven Brownian motion and electric noise [llj . Equation ([3]) explicitly shows that the 
thermodynamic entropy production comes from the time asymmetry in the more microscopic property of dynamical 
randomness. The non-negativity of the entropy production leads to the principle of temporal ordering according to 
which, in noncquilibrium steady states, the typical paths are more ordered in time than their corresponding time 
reversals [12| . 

The remarkable result we will here report is that the aforementioned formula (J3j> allows us to relate the thermody- 
namic entropy production to the information as it is physically recorded in space inside some information processing 
device, generalizing in this way Landauer's principle. We here consider the process of erasure of statistically correlated 
random bits, that is a sequence of bits o\Gi ■ ■ -a m ■ ■ ■ (with a = or 1). We assume that this sequence is initially 
recorded on some spatially extended support such as a recording tape inside the device. The recorded data are de- 
scribed by some probability distribution p(o\<J2 ' ' ■ o~m) gi ym g the occurrence frequencies of the sequences <xi<J2 • • ■ a m 
in the memory of the device. This probability distribution is general with possible spatial correlations among the bits 
(Ti . The information contained in the sequence is characterized by the quantity 

1 = lim -— y~] p(<7i<7 2 • • •<7 TO )lnp((7ic72 • • -am) (4) 
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This is an entropy per bit of information in the sense of Shannon, which we call the information / per bit of the 
sequence. In the case the bits are randomly distributed with equal probability and independently of each other, the 
information per bit is equal to I = In 2. In general, we have the inequality I < In 2. 

The thermodynamic entropy produced during the process of erasure of the aforementioned sequence can be obtained 
using the formula ([3]) according to the following reasoning. We suppose that one bit is erased every unit time r. Let 
us associate a state u)% with the sequence of bits (• • • a m ■ ■ ■ ai+i(Ji ■ 000 • • • ), as done in Fig. [TJ When viewed forward 
in time, the erasure process transforms the state uii into uii+i- Therefore, the process of erasure does not generate 
dynamical randomness since the outcome is unique every time a bit is erased. Accordingly, the dynamical entropy per 
unit time Jl} vanishes, h — 0, since the probability measure fi takes the unit value for the unique path followed during 
erasure and vanishes for all the other paths. On the other hand, the backward process corresponds to the generation 
of a sequence of bits. This reversed process is not unique and generates dynamical randomness at the rate /i R = I 
given by the information contained in the sequence of bits (see Fig. [T|). Indeed, the sequence of bits {u\(J2 • • ■ cr m ) 
now appears at random with the probability p{a\a 2 ■ ■ ■ cr m ) and the probability measure /2 of this reversed process is 
distributed among several possible time-reversed paths. This observation can be understood by the following example. 
Let us consider a particle in a bistable potential, where the left and right wells correspond to the bits and 1. We can 
slowly deform the potential in order to force the particle to end in the left well, which is equivalent to the erasure of the 
initial bit. However, since any such deformation must pass through a potential with a single minimum, undoing the 
latter transformation will result in the particle being in the left or right well with equal probability. This is the analog 
of our backward process, where the sequences of bits are now generated according to their respective probabilities. In 
this way, we can understand how dissipation is closely related to logical irreversibility [l[ . Finally, we infer from Eq. 
Q that the thermodynamic entropy production of the erasure is given by 

AiS = k B {h n - h) = k B I per bit . (5) 

Landauer's principle is recovered in the particular case of statistically independent random bits of equal probability 
for which AiS = k B I = k B In 2 @. This shows that Landauer's principle can be understood from concepts uniquely 
based on dynamical randomness. Therefore, it is completely model independent and k B I is the minimal dissipation 
one can achieve for correlated random bits. Another way to obtain this result is to note that there exist universal 
coding schemes that will asymptotically compress any ergodic sequence of length m to its maximal possible value 772/2 
(the subscript 2 indicates that the information is calculated with logarithms in base 2) [l^]. Once compressed, the 
sequence is composed of uncorrelated random bits with relative probability one half (otherwise it would be possible 
to further compress the sequence, leading to a compression smaller than I2 in contradiction with Shannon's bound 
on data compression) . The usual Landauer principle can then be applied to this compressed sequence which leads to 
a dissipation of k B I2 In 2 = k B I per bit. 

To avoid this computational step in the case where the exact probability distribution p of the bits is unknown, we 
can still obtain the minimal entropy production ^ as follows. The extra cost of compressing data using an a priori 
distribution q instead of the correct distribution p is the relative entropy D(p\\q), given by [13j 

D(p\\q) = lim — Y] p(aia 2 ■ ■ ■ a m ) In P ai<T * (6) 

It has been proven in Ref. [l5| that there exists a stationary ergodic process q with the property to have a vanishing 
relative entropy, D(p\\q) — 0, with respect to any other stationary ergodic process with probability distribution p. 
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Therefore, the erasure process can be done assuming this particular distribution, which results in no extra cost with 
respect to the minimal entropy production ([5]). 
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FIG. 1: Space-time plot of the physical process of erasure of a sequence of bits aia-2 ■ ■ ■ <J m ■ ■ ■ of information. The bits are 
distributed along the space axis on the recording tape of the information processing device. The eraser is located somewhere 
along the recording tape and transforms each bit into a zero in this illustrative example. At every instant of time, the state of 
the system is given by the current sequence of bits: Uj = • • ■ o~j+2<Tj+io~j • 00000 • • ■ where the dot denotes the location of the 
eraser. 

As long as Landauer's principle is equivalent to the second law of thermodynamics [lij], the agreement between 
the argument based on dynamical randomness and the universal compression procedure shows that Eq. (J3j) is the 
appropriate measure of dissipation for general ergodic stationary stochastic processes. 

We notice that the information / is positive if the probabilities p(o \<j2 • • • c m ) characterizing the sequence of bits 
decay exponentially. Nevertheless, information can be sporadically distributed along the sequence of bits, in which 
case the probabilities p(a\a2 ■ ■ ■ cr m ) decay as stretched exponentials so that the information per bit vanishes I —* 
in the l ong -sequence limit m — > 00. The fact is that such sporadic sequences are not uncommon in complex systems 
fit! [l7L Ho. [T9l| . Accordingly, the above considerations lead to the interesting result that dissipation can be arbitrarily 
small during the erasure of sequences of sporadically distributed information. 

In conclusion, we have here obtained a generalization of Landauer's principle on the basis of recent advances in 
nonequilibrium statistical mechanics showing that the second law of thermodynamics can be understand in terms 
of time asymmetry in the property of dynamical randomness [§, [l(| [HI [H| • These advances allow us to relate the 
thermodynamic entropy production during the erasure of a sequence of bits to the information contained in this 
sequence, taking into account the possible statistical correlations among the bits. This result clearly demonstrates 
that the second law of thermodynamics has fundamental implications on the way information is processed in physical 
systems. 
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